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ABSTRACT 

In the San Jose/Evergreen Community College District, 
course and basic skills prerequisites have been established for all 
courses, students with no record of completing these prerequisites 
cannot enroll unless they have been assessed by a counselor, have had 
their prerequisites verified in alternate ways, and have been granted 
override status for enrollment, in spring 1989, 11,307 overrides v;ere 
granted based on completion of equivarent coursew^rk, instructor 
evaluations, life experience, previous degrees, auy. other reasons. A 
study was conducted to compare the academic success of students who 
had completed all course and basic skills prerequisites with that of 
students who had enrolled on override status, and to determine 
whether experience and feedback had improved the counselors' ability 
to appropriately place students. Study findings included the 
following: (1) 69% of the students who had completed all course and 
basic skills requirements (N=35,676) received a grade of A, B, C, or 
Credit, compared to 64% of those who enrolled on override status; (2) 
64% of the counselors successfully placed their students, compared 
with 59% of the counselors in spring 1989; (3) the greatest 
proportion of A and B grades was achieved by students v/ith existing 
transcripts (58%) , students with previous degrees (44%) , and life 
exierience (41%) ; and (4) students who insisted on taking a course 
above their prerequisite level had one of the lowest success rates 
(49%) and the highest course withdrawal rate (33%) . Study findings 
supported the conclusion that counselors who strictly enforce 
prerequisites, use a careful evaluation of relevant life experience, 
and rely on documented evidence of previous coursework can place 
students effectively. (VVC) 
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ABSTRACT 

Counselor Assessment and Student Success for Spring 1989 



by Jon Alan JCangas, Ph.D. 
San Jose/Evergreen Conmunity College District (SJ/ECCO) 
Septenrber 26, 1989 



In 1985, all courses in the SJ/ECCO had course and basic skills prerequisites as appropriate. A computerized prerequisite 
checking system kept track of all students who met prerequisites and all override codes given by counselors. 

The California Ccomunity College Matriculation Plan and Title 5 regulations state that multiple assessment measures are 
needed in the placement of students. Counselor assessment is considered an important part of this process. This research 
examines the questions: (1) Does counselor assessment relate to student success? and (2) Can feedback and experience 
increase counselor success rates? 

The success rate of 69% for all students meeting both course and basic skills prere<,jisites <H = 24,369, excluding override 
codes) for Spring 1989 was tabulated, as was the success rate of 64X for students assessed by counselors (H = 11,307). 
Success was defined as receiving a grade of A, B, C, or CR. 

Codes used included: PD = Previous degree; LX = Life experien^,*:; ET = Existing transcript; PC = Existing coursework; ES = 
Existing test score; IE = Instructor evaluation; OT = Old test score; SE = System error; SI = Student insistence; SO = Some 
other reason; SO = Course to be taken in a sequence, and SW = Student word. 

The results were as followj: 

1. The most consistent predictor of course success over large numbers of students at 69% (H=35,676) was completion of 
course and basic skills prerequisites . 

2. A most significant finding continues to be that counselor Judgment, at its best, can relate to high success and 
that with feedback and experience, counselors can improve their success rates. 

The overall counselor siruess rate was 6A%, up from 59% from Spring 1989. 

Nineteen (19) of 43 (44%) had a 60% or higher success rate, about the same as the 46% for Spring 1989. 

The computer success rate was up from 65% to 69%, making it more difficult for counselors to beat the computer. 
Eighteen percent (18%) beat the computer for Fall 1988 and 7% for Spring 1989. If the computer success rate had 
remained at 65% for Spring 1989, 16% of the counselors would have beat the computer. The percent of counselors 
below 50% was static at 5% for Fall 1988 and 7% for Spring 1989. 

3. Codes indicating the use of existing college transcripts (75%), taking courses in proper sequence (72%), instructor 
evaluation (71%), and the use of previous degrees (64%) were the codes associated with high success. 

4. A significant inprovement in the student word (SW) override code resulted in a 61% success rate, up from 60% in 
Fall 1988 and 51% in Spring 1986. 

5. When an SO (some other reason) code was used to substitute for a prerequisite, a 56% success rate resulted. 

6. The greatest proportion of A and B grades was achieved by students with Existing Transcripts (ET) override codes, 
at 58%, students with Previous Degrees (PD) at 44% and Life Experience (LX) . at 41%. Students meeting all 
prerequisites achieved 35% A's and 3's, 

7. For students who insisted on taking a course (SI) above their assessment/prerequisite level, less than a 50% 
success rate (49%) resulted. Only 24% received an A or B grade , si students also had the highest withdrawal rates 
of all codes, at 33% . 

Counselors who carefully follow prerequisites, use a careful evaluatio>< oi relevant life experience, and rely on docunoented 
evidence of previous coursework can achieve high levels of success for their students . 

Inservice training by counselors who have high success rates for particular override codes will oe used to help train other 
counselors. The hope is that focused feedback on counselor success by code will continue to result in improved student 
success. Further research will evaluate the results. 




COUNSELOR ASSESSMENT AND STUDENT SUCCESS FOR SPRING 1989 



by 

Jon Alan Kangas, Ph.D. 



I. Background 

A* In 1985, the San Jose/Evergreen Community College 

District began its Computerized Prerequisite Checking 
System. All courses in the district were given course 
and basic skills prerequisites appropriate to the 
course. Students who did not have a record of those 
prerequisites in the district's computer could see a 
counselor and have their prerequisites verified or 
established in alternative ways. Counselors gave 
override codes for a particular course to explain the 
way in which the prerequisite had been met. 

B. All California Community Colleges have begun a mandated 
matriculation process which includes assessment, 
orientations, counseling, and follow-up of students. 

We have been advised to use multiple means of 
assessment in making placement decisions rather than 
relying on a single test score. Counselor evaluation 
and judgment have been suggested as important additions 
to the process of assessing and placing students. 

C. The current research has looked at the 11,307 override 
codes given at San Jose City College and Evergreen 
Valley College during Spring 1989, the code used, the 
counselor who used the code, and the success of the 
student in the course for which he/she was given the 
override code. 

These success rates for counselors were compared to the 
success rates of all students district-wide who met 
prerequisites (N = 24,369, excluding students who 
received codes) . 

D. The goal of the project was to discern which override 
codes were associated with the highest and lowest 
success rates and to discover which counselors had 
strategies for the use of codes leading to the highest 
levels of student success. The counselors with high 
success rates will be asked to share their strategies 
during inservice training sessions with other 
counselors. 

E. ""he definition of the override codes used was as 

ollows: 

EC Equivalent coursev/ork completed within the 
District 

EP Experimental purposes (research, etc.) 

ER Discrepancy between computer/student ( Error ) 

ES Existing test score, not in system 

ET Equivalent work seen on transcript 

IE Instructor evaluated student as eligible 
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I*X Life experience meets prerequisites, typically 

judged by faculty 
OT Other test , e.g., old Davis score or other test 

score that relates to our curriculum 
PD Previous degree 

SE System error . Student is OK/DP not OK 

SI Student insisted on taking a course above his/her 
assessment/prerequisite level 

SM Student OK. Course name was changed. 

SO Some other reason. Reason noted on Program 
Planning form 

SQ Secaience . A technical code needed to allow a 

student to enroll in a course in the Summer and 
the following course in the Fall, e.g., English 
one level below lA followed by English lA. The SQ 
is given to the second class in the sequence. 

SK Student ^s word he/she meets prerequisite 
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II. Success by Override Code 



A. Success for the purposes of this study was defined as a 
student's receiving a grade of A, B. C> or CR > 

The success rate for all students in the district who 
met all their prerequisites (excluding those who were 
given override codes) was 69% . up from 65% for Fall 
1988. 

The success rate associated with the various override 
codes (N >10) averaged 64% and ranged from 49% for 
student insistence to 75% for students with existing 
trar scripts . 

The following table and graph indicate the number and 
percentage of students who succeeded for each kind of 
override code given. The flat line across the graph 
indicates the success rate of 69% for students who met 
all prerequisites district-wide (excluding counselor 
overrides) . 
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SUCCESS RATE BY CODE 



EC : 


A'S 




B'S 


C'S 


D'S 


CR'S 




F'S 


W'S 


I'S 


NC'S 


TOTAL 


succ* 


# 


% 


# 


% 


# 


% 


# 


% 


# 


% 


# 

1 


% 




% ' 




% 


# 


% 


# 


% 

100% 


1023 


% 
|60% 


229 


13% 


309 


18% 


265 


15% 


79 


5% 


220 


13% 


87 




5% 


438 


26% 


16 


1% 


70 


4% 


1713 


EP 

ER : 


5 


15% 


12 


36% 5 


15% 


0 


0% 


5 


15% 


4 


12% 


2 


. 6% 


0 


1 

0%; 0 


0% 


33 


100% 


27 j82% 




0% 


0 


0% 0 


0% 


0 


0% 


0 


0% 


0 


0% 


0 f 0% 


0 


0% 


0 


0% 


0 


0% 


0 \ 0% 


ES : 


39 


12% 


33 


10% 39 


12% 


19 


6% 


54 


16% 


19 


6% 


t 

99 ;30% 
i 


7 


2% 


24 


7% 


333 


100% 


165 pox 


ET : 
IE : 


1595 
467 


44% 
12% 


494 
435 


14% ^27 
11% 3G1 


12% j 120 


3% 


204 


6% 


99 


3% 


573 |l6% 


36 


1% 


78 


2% 


3626 


100% 


2720 175% 


8% 


84 


! 2% 


1601 !41% 


95 


2% 


560 


;i4% 


24 


1% 


363 


9% 


3930 


100% 

t 


2804 171% 


LX : 
OT : 


129 


23% j 105 


18% 72 


13% 


17 


3% 


37 


6% 


38 


7% 


155 


27% 


7 


1% 


11 


2% 


1 571 


100% 


343 |60% 

12 167% 

i 

229 j64% 
18 '53% 
519(49% 


6' 


33% 1 3 


17% 3 


17% 


0 


0% 


0 


0% 


2 


11% 


4 


22% 


0 
5 


VA 
1% 


u 
5 
1 

44 


0% 


i 18 


100% 
100% 
100% 
100% 


PO : 


98 27%! 61 


17% 33 


9% 


11 


3% 


37 


10% 


8 


2% 


99 


28% 


l%j 357 
3%1, 34 
4%'' 1057 


SE : 


2 1 6% 1 5 


15% 5 


15% 


1 


3% 


6 


18% 




9% 


11 

350 


32% 
33% 


u 
11 


0% 

1% 


SI : 


113111%; 142 

1 1 


13% 163 


15% 


52 


5% 
0% 


101 

0 


10% 1 81 


8% 


SH : 


0, 0% 

1 J 


0 


0% 0 
16% 601 


0% 


0 


ox\ 


1 


100%! 0 


0% 


0 


0% 


0 


0%! 1 jioo% 


1 

0 0% 


SO : 


699 h6% 


723 


13% 


202 


5% 490 


11% 


230 i 


* 

5%| 1274 
4%| 4 


29% 
9% 


58 
0 


1% 

0% 


176 
3 


4%^ 4453 


100% 


2513 ;56% 


SQ : 


7 15% 


8 


17% 10 


21% 


4 


9%; 


9 


19% 


i 


6% 47 


100%^ 34 72% 


SW : 


291 ^20% 


247 


ir/. 215 


15% 


65 


4% 


147 


10% 


79 j 


5%! 382 

f 


26% 


18 


1% 


33 


2% 1477 


100%! 


900!61% 


TOTAL 


3680 ,21% 


2577 


15% 2139 


1 

12% 

J 


654 


4% 


i 1 
2911 16% 


■ ! 

748 ! 4% 3951 


22% 
20% 


182 


1% 


I 

808 


5% 17650 


100%i 


11307 64% 


DIST 
PRQ HT4 


10566 i20% 


7939 


15% 5999 


I 

i 

ii;; 1771 


3% 


^ y 

11172 f21% 1831 


3% 


10740 


484 


1% 


2331 ^ 


4% 52833 


100% 


i 

35676 « 


o8% 


DIST 
.PRQ HT- 


6886 ,20% 

! 


5362 


15% 3860 


11% 


1117 


3% 


8261 


23% 1083 


3% 


6789 


19%j 


302 


1% 


1523 


4% 35183 


100% 


24369 


69% 



*Succ = A + B .+ C + CR grades; the category of "no grades" was not included in the calculations 

+Total of all students who met prerequisites district-wide. Includes all students who were given override codes 
by counselors. 

•Total of all students who met prerequisites district-wide minus (-) those who had override codes. 
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GRAPH 1 
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Comments 



4 CODES BEAT THE COMPUTER 
Success 

Rate Code Comment 

82% EP This code is designed for experimental 

purposes. It was given in error 33 
times. 

75% ET Counselors are doing an excellent job of 

evaluating transcripts from other 
colleges to establish that students have 
met District prerequisites. This was up 
from 60% for Fall 1983. 

72% SQ This code was used, with a 72% success 

rate , up from 68%, to allow a student to 
take two courses in sequence when the 
first is a prerequisite for the second, 
e.g., English one level below lA in 
Summer and English lA in the Fall. One 
would expect the success rate to 
approximate the computer's 69% success 
rate, which it did. 

71% IE Success associated with instructor 

evaluation, typically in the ESL and 
English areas, was up from 61% for Fall 
1988 to 71%. 



5 CODES WERE AT 60% OR HIGHER 
Success 

Rate Code Comment 

67% OT Students with Old Davis Test scores did 

quite well this year. 

64% PD 64% (down from 69%) of students with 

previous degrees succeeded at their 
coursework. 

61% SW Taking the "student's word" about 

whether they met prerequisites resulted 
in a 61% success rate (up from 60% for 
Fall 1988 and up from 51% in Spring 
1988) , indicating good judgment on the 
part of counselors. 

60% LX The overall success rate of 60% for the 

Life Experience code was down from 69% 
for Fall of 19o<5. This assessment was 
often done by faculty in areas such as 
electronics, laser, computer technology, 
and math, as well as by faculty and 
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counselors in other areas. Previous 
work experience related to a given 
course seems to be a good predictor of 
success 

60% EC 60i (down from 63%) of students 

succeeded who had equivalent coursework 
to a prerequisite in the district. Used 
properly, this means the student met the 
prerequisite by coursework within the 
District. Sometimes the computer may 
not have recognized that an old course 
was equivalent to a new one. It also 
does not pick up repeats, which are 
handled with an override code. 



1 CODF WAS BELOW 50% 
Success 

Rate Code Comment 

49% SI Students insisting on taxin;; courses 

above their prerequisite level had 13^*= 
than a 50% chance of succeeding . 
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III. A and B Grades by Override Code 

A. Students who enter a course with an override code often 
want to be able to achiev^j top grades. The following 
graphs, using data from 'I'able 1, indicate the 
percentage of A and B grades received for each override 
code, excluding EP. 7ne flat line indicates that 35% 
of students who met '^11 course and basic skills 
prerequisites had or B grades. 



GRAPH 2 
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Coiiiinents 



ET, OT, PP. hX, and SW related to high proportions of A 
and B grades compared to the District total of 35%. 

Students with previous college coursework who brought 
in transcripts had achieved 58% A's and B's. Students 
with old Davi^^ test scores had 50%, students with 
previous degrees 44%, related life experience 41%, and 
student's word about previous work related to 36% A's 
and B's. 

These students have generally had previous successful 
college experience. 

SI. IE. ES. and SE are associated with low proportions 
of A's and B's . 

Only 24% of SI students who insist on taxing a coursa 
above their prereguisite level receive A's and B's , 

Of interest is the fact that while instructor 
evaluation (IE) results in placing students in classes 
with high success rates (75%), the nuinber of h's and 
B's by these students is quite low, at 23%. 

ES and SE codes are associated with existing test 
scores not yet downloaded into the computer and with 
system errors. Tliese are codes associated with a 
certain ^?Tiount oi confusion and error and the meaning 
of the resultii are not clear at 'his time. 
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IV. withdrawal Rare by Code 



A. Some codes result in a much greater withdrawal rate 
than others. 



GRAPH 3 
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Coitiments 



Student Insistence fSI) was associated with the highest 
withdrawal rate, at 33% , compared to a 20% withdrawal 
rate for students who met all prerequisites. 

Students with Existing Transcripts (ET) . Instructor 
Evaluation (IE) and who took courses in Seguence (SO) 
had the lowest withdrawal rates , at 16%, 14%, and 9% ^ 
respectively. 

System Error (SE) , Existing Test Score (ES) , and Some 
Other Reason (SO) were associated with confusion and 
with borderline cases and had high withdrawal rates, at 
32%, 30% and 29%, respectively. 



V, Success Rates by Override Code and by Counselor 



A» Table 2 indicates the percentage of students succeeding 
for each override code and for each counselor (01 = 
Counselor #1 through C48 = Counselor #48)* 
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B* Graph 4 plots the success rates for students receiving 
'SO' codes, brok<-^n down by counselor . Counselors who 
did not use the code laore than J 0 times were not 
included* 
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Graph 5 plots the success rates for students receiving 
'oW' codes, broken down by coui:tselor > Counselors who 
did not use the code more than times were not 
included. 
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D. Graph 6 plots the success rates for students receiving 
'ES' codes, broken down by counselor. Counselors who 
did not use the code more than 10 times were not 
included. 
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£• Graph 7 plots the overall success rates for students 
receiving override codes, broken down by counselor . 
Counselors who did not use codes more than 10 times 
were not included. 



GRAPH 7 



y. succ. X couNS. 

o 



80 




Counselor' Nunbex^ 



ERIC 



-16- 

19 



F. Comments 



ES Code ; The range of success for counselors using ES 
(an existing test score was seen that should allow 
the student to qualify for the course) was from 
27% to 69%, down from 53% to 73% for Fall 1988 • 

• None of the counselors achieved a higher 

success rate than the 69% rate for students 
meeting all prerequisites. 

One of 10 counselors (10%) achieved a rate at 

or above 60%, lower than the 20% for Fall 1988. 

Six of 10 counselors (60%) achieved a rate 

below 50%. 

The use of this code relies on a solid knowledge 
of prerequisites and a solid interpretation of 
test scores. It has historically had the greatest 
confusion of any code used and continues to be a 
problem area. 

SO Code ; The success for counselors using the SO code 
(the student meets a prerequisite for some other 
reason) ranged from 41% to 92%, up from the 24% to 
82% associated with Fall 1988. 

Four of 34 counselors (12%) had higher success 

rates than the students who met all 
prerequisites (69%). This was down from the 
18% for Fall 1988. 

Eleven of 34 counselors (32%) were at or above 

60%, up from 27% for Fall 1988. 

Six of 34 counselors (18%) achieved a rate 

below 50%, doing less well than the 12% for 
Fall 1988. This code has a greater number of 
low success rates than most other codes. 

It is clear that a careful examination of the 
reasons why a student may meet a prerequisite not 
included in the computer's information files or 
program can result in success rates higher than 
for students who meet all prerequisites for their 
courses. 

SW code ; The range of success for counselors using the 
SW code (student word that they had the equivalent 
of a prerequisite course elsewhere) was from 36% 
to 83%. 

Six counselors of 25 (24%) had success rates 

higher than the 69% rate of students who met 
prerequisites . 
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Thirteen (13) of 25 counselors (52%) were at or 

above 60%, up from 47% for Fall 1988 • 

Five of 25 counselors (20%) using more than ten 

SW codes had a success rate below 50%, more 
than the 7% for Fall 1988. 

A student's word was not particularly a good 
predictor of success in Spring 1988. For Fall 
1988, however, there was a much more thoughtful 
application of this code by counselors, increasing 
this success rate to an exceptional 60% overall. 
At 61% for Spring 1989, this code has been a 
surprisingly good one. 

Overall Counselor Success Rates ; The overall success 
rate of counselors who used more than 10 codes 
ranged from 44% to 99%. The collective success 
rate increased slightly, from 57% in Spring 1988 
to 59% in Fall 1988 and to 64% in Spring 1989. 

For three of 43 counselors (7%) , their overall 

success rate exceeded the 69% success rate of 
students who had met all prerequisites. 

Nineteen of 43 (44%) with more than ten codes 

were at or above 60% overall, similar to the 
46% for Fall 1988. 

Three of 43 (7%) had an overall success rate 

below 50%. This rate was similar to Fall 1988 
(5%). 

— -The 99% was associated with a very short term 
program in which all students received A 
grades . 

SI code (a new code for Fall 1988): The range of 

success for counselors using the SI code was from 
26% to 68%. 

No counselor using SI exceeded the 69% rate of 

students who met prerequisites. 

Four of 20 counselors (20%) had success rates 

at or above 60%. 

Nine of 20 counselors, or 45%, had success 

rates lower than 50%. 
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General Comments : Several codes and their success 

rates were singled out for review over time. It 
was gratifying and encouraging to note that the 
percentage of counselors with a success rate above 
60% increased or remained stable for two codes as 
summarised below: 

% of Counselors Above a 60% Success Rate 

SP88 F88 SP89 

Overall 32% 46% 44% 

SW 25% 47% 52% 



Several codes needing improvement were represented 
in the percentage of counselors with success rates 
belov: 50% as summarized below: 

% of Counselors Below a 50% Success Rata 



SP88 F88 SP89 



Overall 15% 5% 7% 

ES 43% 33% 60% 

SO 33% 12% 18% 

SW 50% 7% 20% 



ERIC 



-19- 

22 



VI. Summary 



How does counselor judgment as a means of assessment relate 
to student success? Counselors work with an infinite 
variety of information combinations, including test scores, 
previous college coursework in and outside of the district, 
previous degree status, high school grades, work experience, 
appearance, ability to articulate, vocabulary level, 
student's word about educational accomplishments, survey 
information, college grades, home situation, number of hours 
of work, personal support systems, perceived motivation, 
clarity of student goal, and so on* Every student brings a 
different configurations and combinations of the above kinds 
of information. Counselors are faced with the extraordinary 
task of taking each new combination of information, 
weighting the information, and making a unique judament 
about the chances of success for each student. This 
judgment must then be combined with a discussion with the 
student to arrive at a decision about what to do. Student 
variables then get mixed with counselor judgment. This 
study looked at the success rate of this process in 
comparison to the success rate of all students in the 
district who met all course and basic skills prerequisites. 
Counselor judgment resulting in the use of override codes is 
often exercised in adverse circumstances, including 
off -campus sites, short appointment times, long lines, and 
inadequate information from the student. 

Since this first report was done for Spring of 1988, 
counselors have had feedback on their performance on the use 
of codes. ^ It was important to know whether or not feedback 
and experience could influence counselor success rates. 



Documented completion of previous coursework or degrees 
(PD, EC and ET) , along with life experience ^ in related work 
areas and instructor evaluations (LX and IE) , continued to 
be the best ways of predicting success . When these 
variables were analyzed by counselors and faculty, their 
success rates were: 



SP88 F88 SP89 

Existing Transcript 60% 60% 75% 

Sequence N/A 68% 72% 

Instructor Evaluation 59% 61% 71% 

Previous Degree 59% 69% 64% 

Student's Word 51% 60% 61% 

Life Experience 67% 69% 60% 

Existing Coursework 60% 63% 60% 



The most consistent overall predictor of success in courses, 
69% > was completion of course and basic skills prereguisites 
within the District . Basic skills prerequisites in this 
study were met either by completion of basic skills courses 
or by an appropriate test score. Sixty-nine percent (69%) 
of these students were successful. 
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When unique combinations of information had to be combined 
to establish some other reason that a student met a 
prerequisite (and, therefore, should be allowed in a class) , 
the success rate for all counselors was ; 

Some Other Reason 56% 

A student insistence fSI) code was used for the first time 
in Fall 1988 for students who insisted on taking a course 
above their prerecuisite level. This group had one of the 
lowest success rabies at 49% . 

SI students had the highest withdrawal rate of any group, 
at 33% . 

The overall success of all counselors for all codes 
increased from 57% in Spring 1988 to 59% in Fall 1988 and to 
,64% in Spring 1989. 

Again, counselor judgment is exercised along with student 
judgment and it is the result of this interchange that 
results in the override code given. It is evident that one 
cannot generalize about the effectiveness of counselor 
assessment. The judgment and counseling skills in 
relationship to students are exercised more effectively by 
some than others. 

The complexity of the decision making rlealing with an 
incredible range of variables in differing combinations for 
each student can never be encompassed effectively by a test 
or tests. 

However, counselors who carefully follow prerequisites, use 
a careful evaluation of relevant life experience, arid^rely 
on docum<^nted evidence of previous coursework can achieve 
high levels of success at 60% or above . 

All in all, there is great potential for the role of 
counselors in the assessment process and for feedback, 
coupled with practice, to increase success rates. There 
also appears to be the real possibility that with direct 
feedback to a counselor about his or her success rates, the 
counselor's success rate can be increased. It is hoped that 
continual monitoring will see this trend continue. 




-21 





